Network image-synthesis display system

ABSTRACT

Using a plurality of transmission units ( 110, 111, 112 ), multicast transmission of camera images shot by a plurality of cameras ( 101, 102, 103 ) is performed by way of a network, and then, a synthesis-parameter calculation unit ( 105 ) calculates, corresponding to image processing units each, synthesis-parameters required to synthesize using the camera images display images to be displayed on respective display units ( 113, 114, 115, 116 ), and transmits the synthesis-parameters to the image processing units ( 106, 107, 108, 109 ), whereby the image processing units ( 106, 107, 108, 109 ) synthesize display images from the camera images, based on the synthesis-parameters. Synthesis processing of the display images thereby is performed being distributed to the image processing units each.

TECHNICAL FIELD

The present invention relates to a network image-synthesis displaysystem for displaying images shot by a plurality of cameras connected toa network on a plurality of display units connected to the network.

BACKGROUND ART

As devices having a function to display an image shot by aremotely-placed camera, there are a large number of them starting with atelevision. In addition, with the development of a network such as theInternet in recent years, it is generally taking place that an imageshot by a camera connected to the network is displayed on a personalcomputer. Meanwhile, a display system that displays an image in a rangewider than the range one camera can shoot is also proposed bysynthesizing images shot by a plurality of cameras (for example PatentDocument 1). Moreover, in order to display a large image, a large-sizedmultiple screen device that combines images of a plurality of displayprocessing units is also commercialized to display the combined ones asone large image.

[Patent Document 1] Japanese Patent Application Publication No.2002-244683.

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention

However, in a system in which one synthesis image is produced fromimages shot by a plurality of cameras, the synthesis image istransmitted to a remotely-placed display device by way of a network andthe display device displays the received synthesis image, itsimage-synthesis processing becomes a bottleneck in the system operationsbecause a load for the image-synthesis processing is increased accordingto an increase in the number of the cameras. Namely, there is a problemin which the number of camera images that can be processed at the sametime is limited by processing capacity for the image synthesis.

In addition, since the amount of data of a synthesis image increasesaccording to the increase in the number of the cameras, there is also aproblem in which a network load increases at the time of networktransmission of the synthesis image. Moreover, when the display deviceis constituted of a plurality of display units and one synthesis imageis displayed by combining display images of the plurality of displayunits, each of the display units requires processing to segment and/orselect a portion of the received synthesis image in accordance with thedisplay's own region, so that there is also a problem in which aprocessing load of the display units each is also increased.

The present invention has been directed at solving those problems, andan object of the invention is to obtain a network image-synthesisdisplay system that is able to eliminate the bottleneck inimage-synthesis processing caused by the increase in the number ofcameras, and also to reduce network's communication traffic in imagetransmission.

Means for Solving the Problems

A network image-synthesis display system according to the presentinvention comprises: using a plurality of transmission units, multicasttransmission of camera images shot by a plurality of cameras isperformed by way of a network, and then, a synthesis-parametercalculation unit calculates, corresponding to image processing unitseach, synthesis-parameters required to synthesize using the cameraimages display images to be displayed on respective display units, andtransmits the synthesis-parameters to the image processing units each,whereby the image processing units synthesize display images from thecamera images, based on the synthesis-parameters.

EFFECTS OF THE INVENTION

According to the network image-synthesis display system in the presentinvention, image processing units each are so arranged that only adisplay image to be displayed on one display unit undergoes synthesisprocessing, so that, in comparison with a case in which images to bedisplayed on all of display units undergo synthesis processing by oneimage processing unit, there exist effects in which a processing load isnot concentrated and a bottleneck does not occur in the synthesisprocessing. In addition, because it is not necessary to transmit asynthesis image over a network, there also exists an effect that anetwork load can be reduced.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a configuration of a networkimage-synthesis display system according to Embodiment 1 of the presentinvention;

FIG. 2 is a diagram illustrating correlation between camera images shotby a plurality of cameras;

FIG. 3 is a chart showing the operation flows of a synthesis-parametercalculation unit according to Embodiment 1 of the present invention;

FIG. 4 is a diagram illustrating camera images of each of the camerasafter having undergone an image compensation process;

FIG. 5 is a diagram illustrating common image regions between each ofthe camera images;

FIG. 6 is a diagram illustrating an example of specifying image regions;

FIG. 7 is a diagram illustrating an example of segmenting an imageregion;

FIG. 8 is a diagram illustrating one example of correspondencerelationships between display regions and the camera images;

FIG. 9 is a diagram indicating specified contents of asynthesis-parameter;

FIG. 10 is a chart showing the operation flows of an image processingunit;

FIG. 11 is a diagram illustrating a method of synthesizing images in theimage processing unit;

FIG. 12 is a diagram illustrating all of the images displayed on displayunits after having undergone the synthesis therefor;

FIG. 13 is a chart showing the operation flows of a synthesis-parametercalculation unit according to Embodiment 2 of the present invention; and

FIG. 14 is a diagram illustrating a configuration of a networkimage-synthesis display system according to Embodiment 3 of the presentinvention.

EXPLANATION OF NUMERALS AND SYMBOLS

“101,” “102,” “103,” designate cameras; “104,” network; “105,”synthesis-parameter calculation unit; “106,” “107,” “108,” “109,” imageprocessing units; “110,” “111,” “112,” transmission units; and “113,”“114,” “115,” “116,” display units.

BEST MODE FOR CARRYING OUT THE INVENTION Embodiment 1

A configuration of a network image-synthesis display system according toEmbodiment 1 of the present invention will be explained referring toFIG. 1. The network image-synthesis display system is constituted of:cameras 101, 102 and 103 each for shooting images; transmission units110, 111 and 112 for performing multicast transmission of the imagesshot by these cameras to a synthesis-parameter calculation unit 105 andto image processing units 106, 107, 108 and 109 by way of a network 104;the synthesis-parameter calculation unit 105 for calculatingsynthesis-parameters to be used for synthesizing using a plurality ofcamera images, and for transmitting the calculated synthesis-parametersto the image processing units 106, 107, 108 and 109 by way of thenetwork 104; the image processing units 106, 107, 108 and 109 forsynthesizing display images based on camera images received from thetransmission units 110, 111 and 112 and on the synthesis-parametersreceived from the synthesis-parameter calculation unit 105; and displayunits 113, 114, 115 and 116 for displaying the synthesized displayimages.

Note that, the network 104 is a network that supports multicasttransmission, and may be of the Internet, a dedicated communicationsline, a LAN, or the like. In addition, the three cameras and the fourdisplay units are used in FIG. 1; however, the number of cameras andthat of display units are not limited to these.

Next, the operations of the network image-synthesis display system inEmbodiment 1 will be explained. FIG. 2 is an example illustrating thecorrelation between the images in which scenery 201 is shot by the threecameras 101, 102 and 103. In FIG. 2( a), an image shot by the camera 101is a camera image 202, an image shot by the camera 102, a camera image203, and an image shot by the camera 103, a camera image 204. As shownin FIG. 2( a), it is presumed that the camera images 202, 202 and 204have partially overlapping image regions with other camera images, andthat the scenery 201 is shot to be covered therewith as a whole.However, because it is difficult in placing cameras to precisely conformmutual position-attitudes between the cameras, each of the camera imageshas generally a different tilting and a scale as shown in the figure.Therefore, when the camera images 202, 203 and 204 are to be displayedon the display units 113, 114, 115 and 116, the boundary regions of eachof the display images are not seamlessly adjoining if the camera imagesare simply displayed in juxtaposition shown in FIG. 2( b). Note that, inEmbodiment 1 it is presumed that the cameras 101, 102 and 103 arefixedly placed, and their settings such as a camera angle are not variedthroughout a shooting period.

The camera 101 outputs the camera image 202 into the transmission unit110. Similarly, the camera 102 outputs the camera image 203 into thetransmission unit 111, and the camera 103, the camera image 204 into thetransmission unit 112.

The transmission unit 110 transforms the camera image 202 being shot bythe camera 101 into a format compatible with network transmission, andperforms multicast transmission of the transformed image to thesynthesis-parameter calculation unit 105 and the image processing units106, 107, 108 and 109 by way of the network 104. Similarly, thetransmission units 111 and 112 each perform multicast transmission ofthe camera images 203 and 204 shot by the respective cameras 102 and 103to the synthesis-parameter calculation unit 105 and the image processingunits 106, 107, 108 and 109.

The synthesis-parameter calculation unit 105 calculatessynthesis-parameters that is to be used by the image processing units106, 107, 108 and 109 when they synthesize display images to bedisplayed on display units 113, 114, 115 and 116 using the camera images202, 203 and 204, and transmits the calculated synthesis-parameters tothe image processing units 106, 107, 108 and 109. Note that, eachcalculated synthesis-parameter corresponds to each of the imageprocessing units, so that only a corresponding synthesis-parameter istransmitted to each of the image processing units. FIG. 3 is a chartshowing the operation flows of the synthesis-parameter calculation unit105. Hereinafter, the operations of the synthesis-parameter calculationunit 105 will be explained referring to the figures.

The synthesis-parameter calculation unit 105 first receives the cameraimages 202, 203 and 204 from the transmission units 110, 111 and 112(image reception process: Step S301). Next, the received camera images202, 203 and 204 are compared with each other, and common portionsbetween each of the camera images are detected (image comparisonprocess: Step S302). Here, a method of detecting the common portionsbetween each of the camera images is not limited; however, as one ofgeneral techniques, there is a method in which feature points of each ofthe images are extracted, and the feature points are checked if they arecommonly found in each of the images. As a specific technique, there isthe SIFT (Scale Invariant Feature Transform).

Next, an image-compensation parameter for compensating the displacementbetween each of the images is derived so that common portions of thereceived camera images can be overlapped without causing displacement(image-compensation parameter generation process: Step S303).

Here, the roles of the image-compensation parameter will be explained.Because the camera images 202, 203 and 204 are shot by the cameras 101,102 and 103 that are different with each other, a scale of an image, atilting thereof, a camera's viewpoint or the like is different, so thatdisplacement is produced if the camera images are to be overlappedwithout modification. For this reason, it is necessary to compensate thecamera images 202, 203 and 204 so that common portions of each of thecamera images are overlapped without displacement. FIG. 4 illustratescompensated images 401, 402 and 403 resulted in compensation from thecamera images 202, 203 and 204, with their states of having the sametilting and the same dimensions for a common object over the respectivecompensated images. The image-compensation parameter is a parameter thatspecifies such an amount of image-compensation corresponding to each ofthe camera images.

Note that, while a specific format of the image-compensation parameterand a derivation technique thereof are not limited, it is generallyknown that, when a distant view, for example, is shot, a planarprojective transformation (homography) can be derived as thecompensation parameter, if common points of four or more can be detectedbetween each of the images.

Next, an image-quality adjustment parameter to adjust image-quality,such as luminance and color, is generated (image-quality adjustmentparameter generation process: Step S304). FIG. 5 illustrates anappearance in which the compensated images shown in FIG. 4 areoverlapped. In FIG. 5, the same items in FIG. 4 are designated by thesame reference numerals. A common image region 501 indicates, when thecompensated image 401 and the compensated image 402 are overlapped, animage region that is common to each other, and a common image region 502indicates, when the compensated image 402 and the compensated image 403are overlapped, an image region that is common to each other. When imagesynthesis proceeds, two compensated images in the common image regions501 and 502 are added together, thereby their luminance differs from thesingle image region portions, so that it is necessary to adjust theluminance. In addition, when the images of the common image regions showdifferent color therebetween clue to the difference in reflection oflight and camera parameters, it is also necessary to adjust colorcomponents. The image-quality adjustment parameter specifies the amountof image-quality adjustment so that unevenness is not produced in theimage quality between the images when the compensated images areoverlapped with each other as described above. The image-qualityadjustment parameter generation process is also called as colorcorrection or blending, and there are various kinds of proposals as forthe adjustment technique itself when the image synthesis is performed.

Next, image regions to be displayed by the display units 113, 114, 115and 116 are specified (display-region specification process: Step S305).As shown in FIG. 5, a shape of the entirety region of a synthesis imagewhich is composed of the compensated images 401, 402 and 403, isgenerally mismatched in its shape when Step S304 ends. Therefore, theshape is usually different from a shape of the entirety of displayregion which is a combined region of the display regions each fordisplaying the images on the display units 113, 114, 115 and 116. Forthis reason, it is necessary to specify image regions to be displayed bythe display units in accordance with a shape of the entirety of theirdisplay regions.

In what follows, the explanation will be made with the presumption thatan overall shape is rectangular when the display regions of the displayunits 113, 114, 115 and 116 are combined. In FIG. 6, two cases of animage region 601 and an image region 602 are illustrated as an exampleof specifying the image regions. Because the shape of the entirety ofthe display regions by the display units 113, 114, 115 and 116 isrectangular, the shapes of the image regions 601 and 602 each are alsorectangular. With the image region 601, because there exist therein suchregions that are not included in any images of the compensated images401, 402 and 403, it is necessary to take such a process that fills inthose regions with a predetermined color such as black. On the otherhand, the image region 602 is an example in which the maximal rectangleis specified for the image region so that the image therein is includedwith either one of the compensated images 401, 402 and 403. In thefollowing explanation, it is presumed that the image region 602 isspecified as a display region.

Next, synthesis-parameters to be transmitted to the image processingunits 106, 107, 108 and 109 are set (synthesis-parameter settingprocess: Step S306). In Step S306, first, the image region 602 specifiedin the display-region specification process S305 is segmented inaccordance with the shapes of the display regions each by the displayunits 113, 114, 115 and 116. Here, as illustrated in FIG. 7, the imageregion 602 is segmented into four congruent and rectangular regions thatare defined as display regions 702, 703, 704 and 705 to be displayed onthe display units 113, 114, 115 and 116, respectively.

Next, determined are correspondence relationships for synthesis betweeneach of the display regions 702, 703, 704 and 705, and the compensatedimages 401, 402 and 403. FIG. 8 is a diagram illustrating one example ofthese correspondence relationships. In this example, it is presumedthat, first, the image to be displayed on the display region 702 issynthesized from the compensated images 401 and 402; the image to bedisplayed on the display region 703, synthesized from the compensatedimages 402 and 403; the image to be displayed on the display region 704,synthesized from the compensated images 401 and 402; and the image to bedisplayed on the display region 705, synthesized from the compensatedimages 402 and 403. And then, determined are partial regions to be cutout from each of the compensated images so that each of the displayregions can be synthesized as required. For example, as for use in thesynthesis of the display region 702, two partial regions, i.e., apartial region 702 a that is a common portion between the display region702 and the compensated image 401, and a partial region 702 b that is acommon portion between the display region 702 and the compensated image402, are specified as the partial regions that are to be extracted fromthe compensated images 401 and 402. Here, the partial regions 702 a and702 b have the overlapping region with each other as illustrated in FIG.8. In addition, calculated also are display positions of the partialregions 702 a and 702 b each in the display region 702.

Next, using the values each having been derived above,synthesis-parameters are set which are used when the image processingunits 106, 107, 108 and 109 perform image-synthesis processing. Each ofthe synthesis-parameters is composed of an “image-selection parameter”that specifies transmission units to transmit camera-images to be takenfor processing and an “image processing parameter” that specifiescontents of the image-synthesis processing when a display image issynthesized from the received camera-images. In addition, the “imageprocessing parameter” is composed of a “region-range parameter” thatspecifies partial regions to be used by the image-synthesis processingamong the received camera images, an “image-transformation parameter”that can be expressed by a transformation matrix for mapping the pointsin the camera-images to the corresponding points in a display region,and an “image-quality adjustment parameter” that compensates luminance,chromaticity and the like. Here, the transformation matrix can becalculated from; the image-compensation parameter that is the amount ofcompensation for a scale and/or a tilting of the image generated in theimage-compensation parameter generation process S303; and correspondencerelationships between each of the partial regions and display regionsspecified by the “region-range parameter.”

As one example, FIG. 9 indicates such a synthesis-parameter that theimage processing unit 106 applies. The image processing unit 106synthesizes a display image to be displayed on the display unit 113using the camera image 202 received from the transmission unit 110 andthe camera image 203 received from the transmission unit 111, so that an“image-selection parameter” of the synthesis-parameter is set as for thetransmission unit 110 and the transmission unit 111, and an “imageprocessing parameter” is set for each of the transmission units 110 and111. Note that a “region-range parameter,” an “image-transformationparameter” and an “image-quality adjustment parameter” are included inthe “image processing parameter.”

Lastly, the synthesis-parameters being set are transmitted to each ofthe image processing units (synthesis-parameter transmission process:Step S306). Namely, the synthesis-parameter for the image processingunit 106 is transmitted to the image processing unit 106. Similarly,synthesis-parameters for the image processing units 107, 108 and 109 aretransmitted to the image processing units 107, 108 and 109,respectively. According to the above, the synthesis-parameters aregenerated for the image processing units each, and are individuallytransmitted to each of the image processing units.

Note that, it is possible to carry out at one time of the systemstartup, calculation of synthesis-parameters and transmission to each ofthe image processing units by the synthesis-parameter calculation unit105 as an initial setting process.

Next, the operations of the image processing units 106, 107, 108 and 109will be explained. FIG. 10 is a chart showing the operation flows of theimage processing units each. Hereinafter, the operations of the imageprocessing units will be explained referring to the chart. Note that,the image processing units 106, 107, 108 and 109 each operate in similaroperation procedures, so that the explanation will be made here for theimage processing unit 106 as an example.

The image processing unit 106 receives a synthesis-parameter to be usedfor the image processing unit 106 transmitted from thesynthesis-parameter calculation unit 105 (synthesis-parameter receptionprocess: Step S1001). The received synthesis-parameter is used for thefollowing image-synthesis processing. Here, it is presumed that thesynthesis-parameter indicated in FIG. 9 is received as thesynthesis-parameter for the image processing unit 106.

Next received are camera images transmitted from the transmission unitsspecified by the “image-selection parameter” of the synthesis-parameter(image reception process: Step S1002). The image processing unit 106receives, according to the synthesis-parameter in FIG. 9, the cameraimage 202 that the transmission unit 110 has transmitted, and the cameraimage 203 that the transmission unit 111 has transmitted.

Next, based on a region specification parameter of the “region-rangeparameter” included in the “image processing parameter” of thesynthesis-parameter, image portions used for the image-synthesisprocessing are extracted from the received camera images (image-regionextraction process: Step S1003). Using the synthesis-parameter in FIG.9, the image processing unit 106 extracts respective partial regionsfrom the camera image 202 received from the transmission unit 110, andfrom the camera image 203 received from the transmission unit 111. FIG.11 is a diagram for explaining the processes after having undergone StepS1003 in the image processing unit 106. FIG. 11( a) shows a partialregion 1101 extracted from the camera image 202, and a partial region1102, from the camera image 203.

Next, using the “image-transformation parameter” included in the “imageprocessing parameter” of the synthesis-parameter, the extracted partialregions 1101 and 1102 are transformed into compensated partial regions1103 and 1104 (FIG. 11( b)) that are of the shapes corresponding to thedisplay region 702 (image compensation process: Step S1004).

Next, the compensated partial regions 1103 and 1104 are subjected toimage-quality adjustment using the “image-quality adjustment parameter”included in the “image processing parameter” of the synthesis-parameter,and are then superimposed with each other to produce a superimposedimage 1105 (FIG. 11( c)) (image superimposition process: Step S1005).

Next, the superimposed image 1105 is outputted as one display image 1106(FIG. 11( d)) into the display unit 113 (image display process: StepS1006).

From then on, by repeating the processes in Steps S1002 through S1007,it is possible to continuously synthesize the display image to bedisplayed on the display unit 113 using the camera images received fromthe transmission units 110 and 111.

In a similar manner to the image processing unit 106, the imagestransmitted from each of the cameras are processed in real time byoperating the respective image processing units 107, 108 and 109 inparallel, and display images are synthesized to be displayed on each ofthe display units 114, 115 and 116.

The display units 113, 114, 115 and 116 display the respective displayimages outputted from the image processing units 106, 107, 108 and 109.According to the above, when the display units 113, 114, 115 and 116 areplaced in juxtaposition as illustrated in FIG. 12, one screenful ofimages which are integrated with each other, can be displayed as awhole.

Note that, in the above explanation, the “image-transformationparameter” of the synthesis-parameter is specified as a transformationmatrix for transforming camera images into display images; however, the“image-transformation parameter” may be defined as an inverse matrix ofthis transformation matrix. In this case, a partial region for a displayregion of the display units each is to be specified as for a“region-range parameter.” For example, in a case of the image processingunit 106, the display region 702 a may be specified as for the“region-range parameter.”

In addition, it may be so configured that an image-compression encodingprocess is provided in the transmission units 110, 111 and 112 each sothat camera images are transmitted in a compressed state with a reducedamount of data, and at the same time, a compressed-image decodingprocess is provided in the synthesis-parameter calculation unit 105 andthe image processing units 106, 107, 108 and 109 each so that the cameraimages are restored by decoding the compressed images received from thetransmission units. According to the configuration described above, anetwork load in the camera-image transmission can be further reduced.

Moreover, instead of individually transmitting synthesis-parameters foreach of the image processing units to the respective image processingunits, it may be adopted that multicast transmission of thesynthesis-parameters for every image processing units is performed atone time to all of the image processing units, and each of the imageprocessing units uses by selecting only a parameter to be used for itsown image-synthesis processing out of the received synthesis-parameters.

In the network image-synthesis display system according to Embodiment 1as described above, each of the image processing units undergoessynthesis processing of an image only to be displayed on one displayunit, so that, in comparison with a case in which one image processingunit performs synthesis processing of the display images for all of thedisplay units, such an effect can be achieved that a processing load isnot concentrated and a bottleneck does not occur against the synthesisprocessing. In addition, because it is not necessary to transmit asynthesized image over a network, there also exists an effect that anetwork load can be reduced.

Embodiment 2

In Embodiment 1, the synthesis-parameter calculation unit 105 operatesto calculate synthesis-parameters only once as an initialization processof the network image-synthesis display system, and the image processingunits 106, 107, 108 and 109 operate to apply the synthesis-parameters tothe image-synthesis processing at all the time-periods during the systemoperations. For this reason, when there is a change in settings such asa change in disposed position of each camera and/or shooting subjectthereof at a time during the system operations, a displacement may occurin the display by the display units 113, 114, 115 and 116 after thechange in the settings if the synthesis-parameters calculated before thechange in the settings are continuously used. On the other hand, anetwork image-synthesis display system in Embodiment 2 is so configuredthat, even when a change in the disposed position of each camera occursduring the system operations, displaying normally a synthesis image canbe carried out. The arrangement of units each constituting the system ofEmbodiment 2 is similar to that in the system configuration inEmbodiment 1 shown in FIG. 1. Thus, focusing on the different points toEmbodiment 1, the operations of Embodiment 2 will be explained below.

In Embodiment 2, it is presumed that a change in settings of the cameras101, 102 and 103 is allowed during image shooting. For example, a changein disposed position of each camera and/or shooting subject thereof maybe come up with. The transmission units 110, 111 and 112 performmulticast transmission of camera images being shot to thesynthesis-parameter 105 and the image processing units 106, 107, 108 and109 in a similar manner to the case in Embodiment 1.

Next, the operations of the synthesis-parameter calculation unit 105will be explained referring to the operation-flow chart in FIG. 13. Theprocesses (Steps S1301 through S1307) to be performed from receivingcamera images until setting synthesis-parameters in thesynthesis-parameter calculation unit 105 are the same as the processesin Embodiment 1 (Steps S301 through S306 in FIG. 3); thus, theirexplanation is omitted.

Next, each of the calculated synthesis-parameters is compared with theimmediately preceding calculated synthesis-parameter already saved in amemory medium (not shown in the figure) inside the synthesis-parametercalculation unit 105 (synthesis-parameter comparison process: StepS1307). Note that, when synthesis-parameters are calculated for thefirst time after a system startup, there exists no immediately precedingsynthesis-parameter, so that the following processes are carried outwithout performing the comparison with the presumption that a comparisonresult is “Coincidence.”

When a result of the synthesis-parameter comparison process is“Non-Coincidence,” there exists a change in the settings in either ofthe camera 101, 102 or 103, so that the calculated synthesis-parametersare saved into a memory medium inside the synthesis-parametercalculation unit 105 (synthesis-parameter saving process: Step S1308).Subsequently, the calculated synthesis-parameters are transmitted toeach of the image processing units 106, 107, 108 and 109(synthesis-parameter saving process: Step S1309). After completing thesynthesis-parameter transmission, the process returns again to the imagereception process S1301.

On the other hand, when a result of the synthesis-parameter comparisonprocess is “Coincidence,” there exists no change in the settings in anyof the cameras 101, 102 and 103, so that the process returns to theimage reception process S1301 without saving the synthesis-parametersand without transmitting them to each of the image processing units.

From then on, Steps S1301 through S1309 are repeatedly executed. Notethat, a repetition period of the processing may be preset in accordancewith the frequency of a change in camera settings. For example, it ispossible to repeatedly process in a constant period such as in everyfive minutes. It is also possible to repeatedly process not in theconstant period, but in accordance with a predetermined time-schedule.

Although the image processing units 106, 107, 108 and 109 performsynthesis processing of the display images based on synthesis-parametersreceived from the synthesis-parameter calculation unit 105, they areoperated so as to perform the synthesis processing using lately receivedsynthesis-process parameters whenever the synthesis-process parametersare lately received after having the synthesis processing started.According to the configuration described above, even when thesynthesis-parameters are modified, it is possible to change a method ofsynthesizing display images in accordance with the parametermodification. Because of these, the display images of the display units113, 114, 115 and 116 are also changed over in accordance with themodification of the synthesis-parameters.

As described above, the network image-synthesis display system accordingto Embodiment 2 is so configured that the synthesis-parametercalculation unit 105 repeatedly recalculates synthesis-parameters, thecalculated synthesis-parameters are compared with savedsynthesis-parameters, and when the calculated synthesis-parametersdiffer from the immediately preceding synthesis-parameters having beentransmitted to the image processing units, the latestsynthesis-parameters are transmitted to the image processing units withthe presumption that a modification of the synthesis-parameters isdetected, so that there exists an effect that a synthesis image havingno displacement can be quickly displayed even when there is a change insettings such as a change in a setting position of the cameras.

Embodiment 3

In Embodiments 1 and 2, the synthesis-parameter calculation unit isconfigured to calculate synthesis-parameters at the time of systemoperations, and to transmit the synthesis-parameters to each of theinformation processing units. On the other hand, a networkimage-synthesis display system in Embodiment 3 is so configured withoutusing a synthesis-parameter calculation unit that a synthesis-parametercalculated by another means can be used and individually set forinformation processing units each, and that each of the informationprocessing units thus performs image synthesis using such asynthesis-parameter.

FIG. 14 is a diagram illustrating a system configuration of the networkimage-synthesis display system in Embodiment 3. Note that, in FIG. 14,the same reference numerals and symbols designate the same items as, orthe items corresponding to, those shown in FIG. 1. Hereinafter, focusingon the different points to Embodiment 1, the operations will beexplained.

First, synthesis-parameters used in each of the image processing units106, 107, 108 and 109 are preliminarily calculated by some means beforethe system startup. As a method to calculate the synthesis-parameters,it may be adopted that, for example, a planar projective transformationis derived from setting values (a scale, a tilting, and the like) of thecameras 101, 102 and 103, and from information on disposed positionalrelationships among the cameras, and then the synthesis-parameters forthe image processing units each are calculated using thistransformation.

The synthesis-parameters thus preliminarily prepared are set intosynthesis-parameter memory units 1401, 1402, 1403 and 1404 that areattached to the image processing units 106, 107, 108 and 109,respectively. Here, the synthesis-parameters to be set into each of thesynthesis-parameter memory units are dedicated parameters used in eachof the image processing units.

As a specific setting method into the synthesis-parameter memory units1401, 1402, 1403 and 1404, a synthesis-parameter file including, forexample, the written synthesis-parameters may be once read out andloaded into the image processing units 106, 107, 108 and 109, from amemory medium that has recorded the synthesis-parameter file, and thesynthesis-parameter file thus read out and loaded is stored into thesynthesis-parameter memory units 1401, 1402, 1403 and 1404,respectively.

After setting the synthesis-parameters, the image processing units 106,107, 108 and 109 each synthesize, based on the synthesis-parametershaving been set in the synthesis-parameter memory units 1401, 1402, 1403and 1404, display images from the camera images received from thetransmission units 110, 111 and 112. Specific processing procedures forthe synthesis can be carried out in a similar manner to the case inEmbodiment 1.

As described above, in the case where, without using asynthesis-parameter calculation unit, synthesis-parameters arecalculated by another means and are stored into the synthesis-parametermemory units each attached to the image processing units, the imageprocessing units can be operated without depending on a technique how toset the synthesis-parameters by the synthesis-parameter calculationunit, and therefore the scheme for executing the image-synthesisprocessing can be diversified. In addition, because the image synthesiscan be performed exactly according to preset synthesis-parameters, thereexists an effect that the image displaying can be realized in a moreintended manner.

Note that, it may be adopted that synthesis-parameters to be preset intoimage processing units each are such synthesis-parameters that are setusing a plurality of patterns, which are then used as thesynthesis-parameters so that those patterns are changed over inaccordance with a preset schedule. Because of these, it is possible torealize such an image-synthesis processing that is capable ofdiversifying more in image-expressions.

INDUSTRIAL APPLICABILITY

The present invention is applicable to systems such as a remote-imagemonitoring or live-image display system that combines images shot by aplurality of cameras connected to a network, and displays the combinedones as one image.

1. A network image-synthesis display system, comprising: a plurality ofcameras; a plurality of transmission units provided corresponding to thecameras, for performing by way of a network, multicast transmission ofcamera images being shot by the cameras; a plurality of image processingunits for receiving the camera images, and for synthesizing theirrespective display images from the camera images; a plurality of displayunits provided corresponding to the image processing units, fordisplaying said display images; and a synthesis-parameter calculationunit for calculating, corresponding to each of the image processingunits, a synthesis-parameter composed of an image-selection parameterdesignating a combination among the camera images from the plurality oftransmission units and an image processing parameter specifying contentsof image-synthesis processing, and for transmitting by way of thenetwork the synthesis-parameter to the image processing units; whereinthe image processing units each perform, based on the image-processingparameter, the synthesis processing of camera images, among those fromthe transmission units, designated by the image-selection parameter, toproduce said display images, whereby the synthesis processing of saiddisplay images is performed being distributed to the plurality of imageprocessing units.
 2. The network image-synthesis display system as setforth in claim 1, wherein the synthesis-parameter calculation unitrepeatedly calculates said synthesis-parameter, and the image processingunits save therein the synthesis-parameter that have been transmitted tothe image processing units, and compare upon calculation,synthesis-parameter with the saved synthesis-parameter, to transmit thecalculated synthesis-parameter to the image processing units when thecalculated synthesis-parameter differs from the immediately precedingsynthesis-parameter transmitted to the image processing units.
 3. Thenetwork image-synthesis display system as set forth in claim 1 or claim2, wherein the synthesis-parameter calculation unit transmits, to eachof the image processing units, corresponding one of synthesis-parametersfor the image processing units.
 4. The network image-synthesis displaysystem as set forth in claim 1 or claim 2, wherein thesynthesis-parameter calculation unit performs multicast transmission atone time to all of the image processing units, of thesynthesis-parameters for the image processing units, and each of theimage processing units selects only a parameter to be used for its ownimage-synthesis processing out of said synthesis-parameters, and thenperforms synthesizing display images.
 5. A network image-synthesisdisplay system, comprising: a plurality of cameras; a plurality oftransmission units provided corresponding to the cameras, for performingby way of a network, multicast transmission of camera images being shotby the cameras; a plurality of image processing units for receiving thecamera images, and for synthesizing their respective display images fromthe camera images; and a plurality of display units providedcorresponding to the image processing units, for displaying said displayimages; wherein the image processing units each include asynthesis-parameter memory unit that stores a synthesis parametercomposed of an image-selection parameter designating a combination amongthe camera images from the plurality of transmission units and an imageprocessing parameter specifying contents of image-synthesis processing;and display images are produced by synthesizing, based on theimage-processing parameter, camera images, among those from thetransmission units, designated by the image-selection parameter storedin the synthesis-parameter memory unit, whereby the synthesis processingof said display images is performed being distributed to the pluralityof image processing units.